Qualitative distances and qualitative description of images for indoor scene description and recognition in robotics

This thesis is focused on reducing the gap between the acquisition of low level information by robot sensors and the need of obtaining high level information for enhancing human-machine communication and for applying logical reasoning processes. To this end, approaches for qualitative and semantic image description and qualitative distance sensor interpretation were developed. Experimentation was carried out on di↵erent robotic platforms showing useful applications

Qualitative Distances and Qualitative Description of Images for Indoor Scene Description and Recognition in Robotics

The automatic extraction of knowledge from the world by a robotic system as human beings interpret their environment through their senses is still an unsolved task in Artificial Intelligence. A robotic agent is in contact with the world through its sensors and other electronic components which obtain and process mainly numerical information. Sonar, infrared and laser sensors obtain distance information. Webcams obtain digital images that are represented internally as matrices of red, blue and green (RGB) colour coordinate values. All this numerical values obtained from the environment need a later interpretation in order to provide the knowledge required by the robotic agent in order to carry out a task. Similarly, light wavelengths with specific amplitude are captured by cone cells of human eyes obtaining also stimulus without meaning. However, the information that human beings can describe and remember from what they see is expressed using words, that is qualitatively. The research work done in this thesis tries to narrow the gap between the acquisition of low level information by robot sensors and the need of obtaining high level or qualitative information for enhancing human-machine communication and for applying logical reasoning processes based on concepts. Moreover, qualitative concepts can be added a meaning by relating them to others. They can be used for reasoning applying qualitative models that have been developed in the last twenty years for describing and interpreting metrical and mathematical concepts such as orientation, distance, velocity, acceleration, and so on. And they can be also understood by human-users both written and read aloud. The first contribution presented is the definition of a method for obtaining fuzzy distance patterns (which include qualitative distances such as near , far , very far and so on) from the data obtained by any kind of distance sensors incorporated in a mobile robot and the definition of a factor to measure the dissimilarity between those fuzzy patterns. Both have been applied to the integration of the distances obtained by the sonar and laser distance sensors incorporated in a Pioneer 2 dx mobile robot and, as a result, special obstacles have been detected as glass window , mirror , and so on. Moreover, the fuzzy distance patterns provided have been also defuzzified in order to obtain a smooth robot speed and used to classify orientation reference systems into open (it defines an open space to be explored) or closed . The second contribution presented is the definition of a model for qualitative image description (QID) based on qualitative models of shape, colour, topology and orientation. This model can qualitatively describe any kind of digital image and is independent of the image segmentation method used. The QID model have been tested in two scenarios in robotics: (i) the description of digital images captured by the camera of a Pioneer 2 dx mobile robot and (ii) the description of digital images of tile mosaics taken by an industrial camera located on a platform used by a robot arm to assemble tile mosaics. In order to provide a formal and explicit meaning to the qualitative description of the images generated, a Description Logic (DL) based ontology has been designed and presented as the third contribution. Our approach can automatically process any random image and obtain a set of DL-axioms that describe it visually and spatially. And objects included in the images are classified according to the ontology schema using a DL reasoner. Tests have been carried out using digital images captured by a webcam incorporated in a Pioneer 2 dx mobile robot. The images taken correspond to the corridors of a building at University Jaume I and objects with them have been classified into walls , floor , office doors and fire extinguishers under different illumination conditions and from different observer viewpoints. The final contribution is the definition of a similarity measure between qualitative descriptions of shape, colour, topology and orientation. And the integration of those measures into the definition of a general similarity measure between two qualitative descriptions of images. These similarity measures have been applied to: (i) extract objects with similar shapes from the MPEG7 CE Shape-1 library; (ii) assemble tile mosaics by qualitative shape and colour similarity matching; (iii) compare images of tile compositions; and (iv) compare images of natural landmarks in a mobile robot world for their recognition

Towards modelling group-robot interactions using a qualitative spatial representation

This paper tackles the problem of finding a suitable qualitative representation for robots to reason about activity spaces where they carry out tasks interacting with a group of people. The Qualitative Spatial model for Group Robot Interaction (QS-GRI) defines Kendon-formations depending on: (i) the relative location of the robot with respect to other individuals involved in that interaction; (ii) the individuals' orientation; (iii) the shared peri-personal distance; and (iv) the role of the individuals (observer, main character or interactive). The evolution of Kendon-formations between is studied, that is, how one formation is transformed into another. These transformations can depend on the role that the robot have, and on the amount of people involved.Postprint (author's final draft

A Qualitative Spatial Descriptor of Group-Robot Interactions

The problem of finding a suitable qualitative representation for robots to reason about activity spaces where they carry out tasks such as leading or interacting with a group of people is tackled in this paper. For that, a Qualitative Spatial model for Group Robot Interaction (QS-GRI) is proposed to define Kendon’s F-formations [16] depending on: (i) the relative location of the robot with respect to other individuals involved in that interaction; (ii) the individuals’ orientation; (iii) the shared peri-personal distance; and (iv) the role of the individuals (observer, main character or interactive). An iconic representation is provided and Kendon’s formations are defined logically. The conceptual neighborhood of the evolution of Kendon formations is studied, that is, how one formation is transformed into another. These transformations can depend on the role that the robot have, and on the amount of people involved.Postprint (published version

On the Rationality of Explanations in Classification Algorithms

This paper is a first step towards studying the rationality of explanations produced by up-to-date AI systems. Based on the thesis that designing rational explanations for accomplishing trustworthy AI is fundamental for ethics in AI, we study the rationality criteria that explanations in classification algorithms have to meet. In this way, we identify, define, and exemplify characteristic criteria of rational explanations in classification algorithms

A Multilevel Road Alignment Model for Spatial-Query-by-Sketch

A sketch map represents an individual’s perception of a specific location. However, the information in sketch maps is often distorted and incomplete. Nevertheless, the main roads of a given location often exhibit considerable similarities between the sketch maps and metric maps. In this work, a shape-based approach was outlined to align roads in the sketch maps and metric maps. Specifically, the shapes of main roads were compared and analyzed quantitatively and qualitatively in three levels pertaining to an individual road, composite road, and road scene. An experiment was performed in which for eight out of nine maps sketched by our participants, accurate road maps could be obtained automatically taking as input the sketch and the metric map. The experimental results indicate that accurate matches can be obtained when the proposed road alignment approach Shape-based Spatial-Query-by-Sketch (SSQbS) is applied to incomplete or distorted roads present in sketch maps and even to roads with an inconsistent spatial relationship with the roads in the metric maps. Moreover, highly similar matches can be obtained for sketches involving fewer roads

A model for colour naming and comparing based on conceptual neighbourhood. An application for comparing art compositions

A computational model for Qualitative Colour Description, named the QCD model, is defined using the Hue, Saturation and Luminance colour space. This model can name rainbow colours, pale, light and dark colours, and colours in the grey scale, and it has been parameterised by participants of a study in two universities in Spain: University Jaume I and University of Sevilla. The relational structure of the QCD model is analysed by means of a conceptual neighbourhood diagram and it is used to formulate a measure of similarity for solving absolute and relative comparisons of qualitative colours. Moreover, a similarity measure between colour compositions, called SimQCDI, is also developed. A survey test on several art compositions is carried out and the results obtained by the participants are analysed and compared to the computational results provided by the SimQCDI. Also, a comparison to the standard RGB Colour Histogram similarity method is carried out, which shows that the proposed similarity is more intuitive and that the results obtained are similar with respect to quantification. Finally, the cognitive adequacy of the QCD model is also analysed.This work was supported by European Commission through FP7 Marie Curie IEF actions under project COGNITIVE-AMI https://sites.google.com/site/zfalomir/projects/cognitive-ami (GA 328763), the Research Centre on Spatial Cognition at the University of Bremen, the Deutscher Akademischer Austausch Dienst (DAAD), Andalusian Regional Ministry of Economy (project SIMON TIc-8052), Spanish Ministry of Economy and Competitiveness (project TIN2011-24147), Generalitat Valenciana (project GVA/2013/135) and Universitat Jaume I (Project P11B2013-29)

Spatial reasoning about qualitative shape compositions. Composing Qualitative Lengths and Angles

Shape composition is a challenge in spatial reasoning. Qualitative Shape Descriptors (QSD) have proven to be rotation and location invariant, which make them useful in spatial reasoning tests. QSD uses qualitative representations for angles and lengths, but their composition operations have not been defined before. In this paper, the Qualitative Model for Angles (QMAngles) and the Qualitative Model for Lengths (QMLengths) are presented in detail by describing their arity, reference systems and operators. Their operators are defined taking the well-known temporal model by Allen (1983) as a reference. Moreover, composition tables are built, and the composition relations of qualitative angles and lengths are proved using their geometric counterparts. The correctness of these composition tables is also proved computationally using a logic program implemented using SwiProlog

Qualitative spatial logic descriptors from 3D indoor scenes to generate explanations in natural language

Falomir Z, Kluth T. Qualitative spatial logic descriptors from 3D indoor scenes to generate explanations in natural language. Cognitive Processing. 2018;19(2):265-284.The challenge of describing 3D real scenes is tackled in this paper using qualitative spatial descriptors. A key point to study is which qualitative descriptors to use and how these qualitative descriptors must be organized to produce a suitable cognitive explanation. In order to find answers, a survey test was carried out with human participants which openly described a scene containing some pieces of furniture. The data obtained in this survey are analysed, and taking this into account, the QSn3D computational approach was developed which uses a XBox 360 Kinect to obtain 3D data from a real indoor scene. Object features are computed on these 3D data to identify objects in indoor scenes. The object orientation is computed, and qualitative spatial relations between the objects are extracted. These qualitative spatial relations are the input to a grammar which applies saliency rules obtained from the survey study and generates cognitive natural language descriptions of scenes. Moreover, these qualitative descriptors can be expressed as first-order logical facts in Prolog for further reasoning. Finally, a validation study is carried out to test whether the descriptions provided by QSn3D approach are human readable. The obtained results show that their acceptability is higher than 82%

Special issue on logics and artificial intelligence

There is a significant range of ongoing challenges in artificial intelligence (AI) dealing with reasoning, planning, learning, perception and cognition, among others. In this scenario, many-valued logics emerge as one of the topics in many of the solutions to some of those AI problems. This special issue presents a brief introduction to the relation between logics and AI and collects recent research works on logic-based approaches in AI